Neurons That Confuse Mirror-Symmetric Object Views

Neurons in inferotemporal cortex that respond similarly to many pairs of mirror-symmetric images – for example, 45◦ and −45◦ views of the same face – have often been reported. The phenomenon seemed to be an interesting oddity. However, the same phenomenon has also emerged in simple hierarchical models of the ventral stream. Here we state a theorem characterizing sufficient conditions for this curious invariance to occur in a rather large class of hierarchical networks and demonstrate it with simulations. Neurons in inferotemporal (IT) cortex – the highest visual area in the ventral visual pathway – respond with varying degrees of selectivity to a specific pattern such as one particular view of an object. In some cases, single cells responding in a similar way to many pairs of images, each representing an object view and its mirror-symmetric counterpart, have been reported (Logothetis et al., 1995), as shown in fig. 1. Similar tuning has also been observed for some units in a hierarchical model of cortex, as shown in fig. 2. This property seems puzzling and so far appeared to originate from an accidental tuning of neurons in IT to features in common between views and their mirror-symmetric versions. Without discounting this possibility, we focus here on establishing an alternative – and quite possibly, complementary – explanation. We argue that it may (also) reflect a property of a large class of hierarchical architectures. Recently, we have been developing a general theory (Smale et al., 2010) of hierarchical feedforward architectures such as the model of visual cortex shown in fig. 3. Within this theoretical framework we have established